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Abstract 

Today's increasing demand for wirelessly uploading a 
large volume of User Generated Content (UGC) is still 
significantly limited by the throttled backhaul of res- 
idential broadband (typically between 1 and 3Mbps). 
We propose BaPu, a carefully designed system with 
implementation for bunching WiFi access points' back- 
haul to achieve a high aggregated throughput. BaPu is 
inspired by a decade of networking design principles and 
techniques to enable efficient TCP over wireless links 
and multipath. BaPu aims to achieve two major goals: 
1) requires no client modification for easy incremental 
adoption; 2) supports not only UDP, but also TCP traf- 
fic to greatly extend its applicability to a broad class 
of popular applications such as HD streaming or large 
file transfer. We prototyped BaPu with commodity 
hardware. Our extensive experiments shows that de- 
spite TCP's sensitivity to typical channel factors such 
as high wireless packet loss, out-of-order packets arrivals 
due to multipath, heterogeneous backhaul capacity, and 
dynamic delays, BaPu achieves a backhaul aggregation 
up to 95% of the theoretical maximum throughput for 
UDP and 88% for TCP. We also empirically estimate 
the potential idle bandwidth that can be harnessed from 
residential broadband. 

1. INTRODUCTION 

Nowadays, the mobile devices are equipped with high- 
resolution cameras and a variety of sensors and are 
quickly becoming the primary device to generate per- 
sonal multimedia content. Both the quality and quan- 
tity of User Generated Content grows continuously. This 
naturally leads to end users' ever increasing demand of 
sharing these high volume of UGC with others in an 
instant way. Prominent examples of services allowing 
multimedia content sharing are YouTube, Daily mot ion, 
and various social networking platforms like Facebook 
and Google+. In addition, there is also a trend of in- 
stantly backing up personal files in the "Cloud Storage", 
such as Dropbox and iCloud. To obtain a satisfactory 
user experience, users need sufficient uplink bandwidth 
to do the fast bulk data transfer to the Internet. How- 



ever, today's ISP's generally offer highly throttled up- 
link bandwidth around 1 to 3Mbps. As a result, in- 
stant sharing of HD content or fast data backup in 
the "Cloud" is generally infeasible in today's residen- 
tial broadband. For example, iPhone 5 takes video at 
1080p and 30fps, which translates to around 200MB per 
minute. With 3Mbps uplink, it takes over an hour to 
upload a 10 minute video clip to iCloud! These lim- 
itations are even more critical for users who desire to 
retain the control over their content and intend to share 
them directly from their homes. This calls for solutions 
to scale backhaul uplink. 

In this work, we propose a complete software based 
solution on WiFi Access Point for aggregating multiple 
broadband uplinks, with the assistance of the WiFi in- 
frastructure in the same neighborhood. Our solution 
features complete transparency to client devices and 
high aggregated throughput for both TCP and UDP, 
even in lossy wireless environment. Our work is pri- 
marily motivated by the following observations: 

• Asymmetric WiFi and broadband capacity: 

In contrast to the broadband uplink, WiFi has a 
much higher bandwidth. 802.1 In supports up to 
600Mbps data rate. With sufficiently high WiFi 
bandwidth, it is beneficial to wirelessly commu- 
nicate with multiple proximate APs and "harness" 
the idle broadband uplink bandwidth behind them. 

• Mostly idle broadband uplinks: Since Febru- 
ary 2011, we have developed and deployed a WiFi 
testbed [24] in Boston urban area, aiming to moni- 
tor the usage pattern of residential broadband net- 
works. As shown in Table 1, this testbed con- 
sists of 30 home WiFi APs running customized 
firmware based on OpenWRT [25]. Each AP re- 
ports the network statistics every 10 second. Dur- 
ing a 18 month period, we have collected over 70 
million records. We observe that the broadband 
uplink utilization is very low. Figure 1 shows the 
probability of uplink bandwidth being consumed 
at most certain value during a 24 hour time win- 
dow. Throughout the day, there is at least 50% 
chance that uplink is completely idle. Even dur- 
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Location 

Home APs 

Data collection time 

Network stats samples 



Boston urban area 
Comcast (26), RCN (4) 
Feb. 2011 - Dec. 2012 
70 million 



Table 1: Data summary of Broadband usage statistics 
collected from residential WiFi testbed 



ing peak hours, there is over 90% chance that the 
uplink bandwidth usage is below 100Kbps. This 
implies that there exists a considerable amount 
of idle uplink bandwidth resources, which makes 
bandwidth harnessing through neighboring APs a 
viable approach for scaling the uplink capacity. 
WiFi densely deployed in residential area: 
The density of WiFi APs is very high in residential 
areas. Already in 2005, authors in [2] measured 
more than 10 APs per geographical location. Re- 
cently, we conducted Wardriving measurements in 
4 urban residential areas in Boston. Our results 
(Table 2) indicate 3192 unencrypted WiFi APs, 
accounting for 14.2% of total APs detected dur- 
ing our wardriving. As shown in Figure 2, there 
are on average 17 APs available at each location, 
with on average 7 to 12 APs on each channel. This 
enormous presence of WiFi APs also justifies the 
feasibility of the concept of bandwidth aggregation 
via open APs. 



Total APs 
Unencrypted APs 



22,475 (100%) 
3,192 (14.2%) 



Table 2: Boston Wardriving Data Summary 

WiFi becoming open and social: Nowadays, 
end users have an ever increasing demand of ubiq- 
uitous Internet access. Driven by such demand, 
there is an emerging model that broadband sub- 
scribers host two WiFi SSIDs, one encrypted for 
private use, the other unencrypted to share part 
of their bandwidth as public WiFi signal to mobile 
users for free or for some payment in return. Unen- 
crypted guest- WLAN is now a standard feature in 
mainstream home WiFi AP products like LinkSys 
and D-Link. FON [9], a leading company in this 
area, claims to have over 7 million hotspots world- 
wide. In addition, WiFi APs are quickly becoming 
cloud-managed devices, such as FON and Meraki 
[19]. AP firmware updates are regularly pushed 
from the Cloud. Given such trend of WiFi quickly 
becoming social and cloud-powered, we believe a 
software based solution on WiFi AP can make a 
quite easy incremental adoption of our technology. 
Lack of efficient and practical solution: De- 
spite a set of prior work exploring how to aggre- 
gate wired bandwidth through WiFi, they either 
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Figure 1: CDF of uplink bandwidth usage (per house- 
hold) in residential broadband. 
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Figure 2: Available APs per scanning in Wardriving. 

require heavy modification on client, or support 
only specific application, such as UDP based bulk 
file transfer [15]. Our goal is to design a client 
transparent, software based solution, which is easy 
to deploy and offer generic support for both TCP 
and UDP applications. 

With the above motivations and goals bearing in mind, 
we design our BaPu system. Our major contributions 
in this work are summarized as follows: 

Transparency to client: BaPu does not require 
any modification to client devices. The client device, 
running in Station mode, transfers data via unicast link 
to its "home" AP. Given the broadcast nature of wire- 
less communication, such unicast packets can be "heard" 
by both "home" AP and the "neighboring" APs on the 
same channel. They each upload a share of received 
(overheard) packets to the destination in a collabora- 
tive manner. Such transparency to client devices allows 
all kinds of wireless client devices and a broad class 
of legacy network applications, such as streaming and 
large file transfer, to seamlessly utilize BaPu system. 

Efficient aggregation for both TCP and UDP: 
Given our design goal of client transparency, some com- 
monly adopted technique in the existing bandwidth ag- 
gregation solutions, such as parallel TCP flows [16], are 
no longer valid, because it requires client applications 
to intentionally establish multiple connections through 
different APs and transfer data in parallel. The mul- 
tiplexing of a single TCP flow through multiple paths 
raises many technical challenges which makes efficient 
aggregation non-trivial. Our initial approach relied on 
coding across paths, however, we could show that a con- 
ceptually simpler mechanism, which we call Proactive- 
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Figure 3: BaPu system architecture and two example 
application scenarios. Scenario 1 (left): Sender 1 shares 
an HD video with a remote end user through a BaPu- 
enabled Home-AP and neighboring Monitor-APs. Sce- 
nario 2 (right): Sender 2 backs up a large file to iCloud 
through a BAPu-enabled Home-AP and Monitor-APs. 

ACK, combined with a reliable 802.11 unicast to the 
"home" AP, and adequate scheduling are sufficient. 

Prototype with commodity hardware: We pro- 
totyped our complete BaPu system on commodity WiFi 
APs. We flash Buffalo 802. lln APs with Linux based 
OpenWRT [25] firmware. As of today, OpenWRT sup- 
ports devices by more than 50 manufacturers and hun- 
dreds of models. This gives us a great selection of com- 
patible devices to deploy BaPu. 

Evaluation: We have conducted an extensive set of 
experiments to evaluate BaPu in various realistic net- 
work settings. Our results show that BaPu achieves 
high aggregated throughput in UDP transmission, har- 
nessing over 95% of total uplink bandwidth. With the 
Proactive- ACK mechanism, BaPu harnesses over 88% 
of total uplink bandwidth in TCP transmission. 

Design guideline for bandwidth sharing: We 
propose a simple traffic shaping method on AP, which 
allows us to harness as much idle bandwidth as possible 
without affecting home users' regular network usage. 
We also give an estimation of idle uplink bandwidth 
under a typical residential broadband usage pattern. 

Our paper is organized as follows. We first present 
an overview of BaPu system. The details of our design 
is discussed in Section 3. We evaluate the performance 
of BaPu in Section 4. In Section 5, we quantitatively 
evaluate the potential impact of uplink sharing to home 
users' regular network usage. We discuss related work 
in Section 6 and conclude the paper in Section 7. 

2. SYSTEM OVERVIEW 

2.1 Application Scenarios 

For ease of understanding, we first introduce two typ- 
ical application scenarios that benefit from BaPu- see 
Figure 3. 



Scenario 1: Instant Sharing of HD Video: In 

order to retain the control of personal content, Sender 1 
shares his HD video directly from his hard drive and 
streams it instantly, i.e., in real-time, to the other user 
- Destination 1. Both users are connected to their 
Home-APs, with an uplink connection from Sender 1 
to Destination 1 throttled to 1 ~ 3Mbps by Sender l's 
ISP. The HD video has 8Mbps playback rate (standard 
1080p video bit rate), so Sender l's single uplink cannot 
handle this content in real-time. However with BaPu, 
the idle uplink of the neighboring Monitor-APs are ex- 
ploited to boost the uplink throughput. The BaPu- 
Gateway, the Home-AP of Destination 1, plays the role 
as the aggregator to receive and forward multiplexed 
traffic to Destination 1. 

Scenario 2: Instant Backup of Large File: Sender 2 
wishes to backup his HD video clip to some cloud stor- 
age service such as iCloud. With the 3Mbps uplink rate, 
it takes over an hour to upload a 30 minute HD video. 
With BaPu, neighboring Monitor-APs and Home-AP 
upload data in parallel. iCloud just needs to deploy 
a gateway server in front of the cloud storage servers. 
This gateway server runs the BaPu- Gateway software 
to handle parallel uploading from multiple paths. Using 
BaPu, file uploading time is greatly reduced. 

Security and Privacy: In both application scenar- 
ios, the APs are configured to have two SSIDs, an en- 
crypted SSID (i.e., WPA) for private use, and an unen- 
crypted (open) SSID. The BaPu traffic is carried over 
neighbouring unencrypted SSIDs with end-to-end secu- 
rity (e.g., SSL/TLS), while allowing the main SSID of 
participating APs to remain encrypted. 

2.2 BaPu Description 

First, we introduce the notation used in this paper: 

• Sender: device uploading data to a remote desti- 
nation. 

• Receiver: device receiving uploaded data from a 
remote sender. 

• Home-AP: AP which Sender is associated to. 

• Monitor- AP: in physical proximity to Receiver and 
Home-AP, a couple of neighboring APs, the Monitor- 
APs, run in monitor mode on the same WiFi chan- 
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Figure 5: BaPu Protocol Traffic Flow. The ACKs (red 
color) are managed for TCP only. 

nel as Home-AP. 

• BaPu -Gateway: gateway device connected to Re- 
ceiver. As shown in Figure 3, BaPu- Gateway can 
be the WiFi AP of the Receiver or the gateway 
server at the edge of some cloud data center. 

• BaPu-t4P: for abstraction, Home-AP and Monitor- 
AP will be called BaPu-^4P, thereby representing 
the role that APs play in a BaPu data upload 
session. 

In BaPu, Sender is associated with its Home-AP, 
and the uploading of data is aggregated via unencrypted 
wireless link. The data, however, are protected with 
some end-to-end security mechanism (e.g., SSL/TLS). 
Home-AP and Monitor-AP are configured to run in 
both WiFi AP mode and WiFi monitor mode 1 . The 
general BaPu-^4P setup is illustrated in Figure 4. 

The WiFi link between a Sender and its Home-AP 
generally provides high bandwidth, up to hundreds of 
Mbps with 802. lln. The link between a BaPu-^P and 
the Receiver, however, is throttled by the ISP. At the 
remote end, we place a BaPu- Gateway immediately be- 
fore the Receiver. The connection between the BaPu- 
Gateway and the Receiver is a wired or wireless high- 
speed link. Note that being in physical proximity, uni- 
casts between Sender and Home-AP (AP mode) can be 
overheard by (some of) the neighboring Monitor- APs 
(monitor mode). 

At a high level, BaPu is a centralized system with the 
controller residing at BaPu- Gateway. BaPu provides 
an aggregation of uplink bandwidth that is carried out 
as follows. 

1. Sender starts a TCP or UDP upload session to 
Receiver through its Home-AP via WiFi. 

2. Home-AP and Monitor-AP overhear WiFi pack- 
ets and identify if this can be a "BaPu " session by 
checking the destination IP and port. In our pro- 
totype described in Section 4, we choose a specific 



9 1 Modern WiFi drivers, such as the prominent Ath9k 
family, allow one physical WiFi interface to support running 
in multiple modes 



UDP/TCP port for all traffic that allows band- 
width aggregation. 

3. BaPu-t4Ps register themselves as a contributor to 
BaPu -Gateway. 

4. In BaPu, Home-AP and Monitor-AP collaborate 
to upload data for Sender, following a schedule 
that is determined by BaPu -Gateway . We will 
explain this scheduling mechanism and protocol 
details later in Section 3. 

Practically speaking, Home-AP and Monitor-AP 
now capture Sender's packets with libpcap from 
the monitor mode, and stores them in a buffer. 

5. For each packet overheard, Home-AP and Monitor- 
AP send packet reception reports to the BaPu- 
Gateway. 

6. For an UDP session, on reception of the reports, 
BaPu- Gateway determines which BaPu-^4P will 
forward the captured packet in step 8. A schedul- 
ing message is then prepared to include the se- 
lected BaPu-t4P's identity, and this scheduling 
message is broadcast back to all BaPu-^4Ps par- 
ticipating in the current session. 

7. A TCP session is much more challenging to sup- 
port than UDP. To properly multiplex Sender's 
single TCP flow through multiple paths, BaPu- 
Gateway adopts a mechanism we call Proactive- 
ACK mechanism. BaPu- Gateway sends spoofed 
TCP ACKs to Sender as the BaPu session goes 
on. Proactive- ACK is designed to make BaPu 
work efficiently with legacy TCP congestion con- 
trol. We will explain details later in Section 3. 

8. The scheduled AP forwards the buffered packets 
to BaPu- Gateway, which forwards to Receiver. 

9. Any downlink traffic from Receiver to Sender just 
follows the default network path, i.e., from Re- 
ceiver over BaPu- Gateway and Home-AP to Sen- 
der. 

Figure 5 shows BaPu's protocol flow. In the follow- 
ing section, we now discuss BaPu's research challenges 
in details and give an insight to our design decision at 
each step of the protocol. 

3. BAPU 

In this section, we describe the whole BaPu sys- 
tem in details. We discuss technical challenges aris- 
ing and propose solutions to achieve an efficient and 
practical aggregation system. We remark that BaPu 
shares some similarities in the high-level architecture 
with prior work (e.g., Link-alike [15], FatVAP [16]), 
which presented neat systems for aggregating the band- 
width among APs. However, from the practicality as- 
pects, the applicability of those systems is still limited 
due to constraints such as heavy modification of client 
devices or support for only specific applications (e.g., 
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large file transfer). Yet our ultimate goals of the trans- 
parency for the users, and the high-throughput trans- 
mission for all kinds of user applications require a new 
solution with unique characteristics. 

3.1 Network Unicast 

First, the transparency goal requires that legacy trans- 
port protocols be usable for data transmission from 
Sender to Receiver. Accordingly, the Sender must be 
able to transmit data to the Receiver via network uni- 
cast through its Home-AP. The second reason for the 
need of network unicast is to increase the reliability of 
the transmission, because BaPu supports TCP, whose 
performance depends on the reliability of the underly- 
ing MAC layer. To be clearer, according to the IEEE 
802.11 standard, a packet with a broadcast destination 
address is transmitted only once by the WiFi device, 
whereas up to 12 MAC-layer retransmissions are tried 
for a failed unicast destination address, therefore a uni- 
cast is much more reliable than a broadcast. Conse- 
quently, supporting network unicast is an essential re- 
quirement in BaPu, while in prior work [15], broadcast 
is preferred due to the simplicity goal of their system. 

Packet Overhearing: In WiFi network, although both 
network unicast and network broadcast use the same 
method of wireless broadcast to transfer data, the differ- 
ence lies in the MAC layer, where the next-hop physical 
address is specified to the unicast address or broadcast 
address. This complicates the packet overhearing capa- 
bility at Monitor-APs. As Home-AP is the first hop 
in the transmission path, the Sender, according to the 
underlying routing protocol, has to use the Home-AP's 
physical address as the next-hop address in the 802.11 
header. While Home-AP as a next hop can receive the 
packet, Monitor-APs automatically discard the packet 
due to mismatched physical address. Therefore, barely 
relying on the default network behavior does not let 
Monitor-APs capture packets sent by Senders in other 
WLANs. 

BaPu's solution is to configure BaPu-^4Ps to oper- 
ate simultaneously in two modes: AP mode and monitor 
mode. The former mode is used for serving clients in the 
AP's own WLAN, whereas the latter is used for over- 
hearing packets in the air. In monitor mode, packets 
are captured in raw format via the use of libpcap. 

Packet Identification: Each packet sent from the 
Sender (step 1) contains the session information in the 
packet's IP header such as the protocol identifier, the 
source and destination IP addresses and ports. With 
this information, Home-AP can uniquely identify the 
Sender (step 2). In contrast, Monitor-APs may have 
ambiguity in identifying the Sender, as Senders from 
different WLANs may (legally) use the same IP ad- 
dress. To resolve such conflict, we write a frame parser 
for the packet's MAC header to obtain the BSSID that 



identifies the WLAN the session belongs to. Therefore, 
any session in BaPu is now uniquely determined on 
the following 6-tuple <BSS ID, proto, srcIP, dstIP, 
srcPort, dstPort>. 

Duplicate Elimination: As mentioned earlier, uni- 
casting a packet may involve a number of (MAC-layer) 
retransmissions due to wireless loss occurred between 
the Sender and its Home-AP. This benefits the data 
transmission between them. Nevertheless, it is possible 
that a nearby Monitor- A P can overhear more than one 
(re) transmission of the same packet, which creates du- 
plicates and floods the Monitor- AP's uplink if all the 
retransmitted packets get scheduled. To identify the du- 
plicate packets, we keep records of IPID field in the IP 
header of each overheard packet. Since IPID remains 
the same value for each MAC-layer retransmission, it 
allows Monitor-APs to identify and discard the same 
packet. It is worth noting that in TCP transmission, 
the TCP sequence number is not a good indicator to 
identify the duplicate packets, as it is unique for TCP- 
layer retransmitted packets, but not unique for MAC- 
layer retransmissions. 

3.2 Tunnel Forwarding 

The transparency goals requires that the Sender's 
data transfer session is unaware of the aggregation pro- 
tocol in BaPu. A seemingly straightforward solution is 
that Home-AP and Monitor-APs forward the Sender's 
packets with spoofed IP addresses. It is, however, im- 
practical for two reasons: 1) many ISPs block spoofed 
IP packets; 2) forwarded packets by Monitor-APs are 
unreliable, because they are raw packets overheard from 
the air. Our approach is that each BaPu-^4P conveys 
the Sender's data via a separate TCP tunnel. Since we 
support a transparency for aggregation over multiple 
paths, the techniques for tunnelling and address resolv- 
ing in each single path require a careful design at both 
BaPu-^Ps and BaPu -Gateway . 

Tunnel Connection: Once a BaPu-^4P identifies a 
new Sender- Receiver session (step 3) based on the 6- 
tuple, it establishes a tunnel connection to BaPu -Gateway. 
Regardless of the session protocol, a tunnel connection 
between the BaPu-^4P and BaPu -Gateway is always 
a TCP connection. The choice of TCP tunnel is par- 
tially motivated by the TCP-friendliness. We desire 
to aggregate the idle bandwidth of BaPu-^4Ps without 
overloading the ISP networks. Besides, since TCP tun- 
nel can provide a reliable channel, it helps keep a simple 
logic for handling a reliable aggregated transmission. 

Forwarding: In the registration (step 3) to BaPu- 
Gateway, the BaPu-^4P receives an APID as its "con- 
tributor" identifier for the new session. The APID is used 
in all messages in the protocol. Both control messages 
(registration, report, scheduling) and data messages are 
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exchanged via the TCP tunnel, which ensures reliable 
transmissions. On reception of a scheduling message 
with matching APID, the Monitor-AP encapsulates the 
corresponding Sender's packet in a BaPu data message 
and sends it to BaPu- Gateway (step 8), which then ex- 
tracts the original data packet, delivers to the Receiver. 
In BaPu, the control messages are short, thus intro- 
ducing only a small overhead in the backhaul. 

NAT: In WiFi network, the Sender is behind the Home- 
AP and the Sender might also reside behind a gateway. 
By default, a NAT (network address translation) is per- 
formed for the session between the Sender and the Re- 
ceiver. In BaPu, the Sender's data are conveyed to 
the Receiver via separate tunnels from each participat- 
ing BaPu-t4P. Therefore, different from the address 
translation in a traditional network, BaPu requires that 
the NAT mapping information of the end-to-end session 
must be known to transfer the embedded data to the de- 
sired Receiver. Consequently, in the registration step, 
each BaPu-^P, besides APID, also receives the NAT 
mapping records from BaPu -Gateway . 

Besides, since the downlink capacity is enormous, we 
allow all reverse (downlink) traffic from Receiver to Sen- 
der to traverse along the default downlink path. In ad- 
dition, as there might be multiple tiers of NAT boxes 
in the middle, we must ensure that the NAT mapping 
for a session is properly installed on all NAT boxes 
along the path between Sender and Receiver in order 
for the returning traffic to traverse the NAT boxes prop- 
erly. Therefore, the first few packets in a session must 
go along the default uplink path. This means the first 
packet in UDP sessions or the 3-way handshake traffic 
in TCP sessions are not tunnelled. 

3.3 TCP with Proactive- ACK 

TCP ensures successful and in-order data delivery be- 
tween the Sender and the Receiver. In TCP, each 
packet is identified with a sequence number and must be 
acknowledged by the Receiver to indicate the proper de- 
livery. The Sender maintains a dynamic CWND (con- 
gestion window) during the on-going session, which in- 
dicates the maximum number of packets that can be 
sent on the fly, therefore determines the TCP through- 
put. 

The Sender's CWND size is affected by the acknowl- 
edgements received from the Receiver. First, the growth 
rate of CWND depends on the rate of receiving acknowl- 
edgements, i.e., the link latency. Second, missing ac- 
knowledgement within a RTO (retransmission timeout) 
causes the Sender to issue a retransmission. On the 
other side, if the Receiver receives some out-of-order 
sequence, it sends a DUPACK (duplicate acknowledge- 
ment) to inform the Sender of the missing packet. By 
default [3], the Sender will issue a fast retransmission 
upon receiving 3 consecutive DUPACKs. Both retrans- 



mission and fast retransmission cause the Sender to cut 
off the CWND accordingly to slow down the sending 
rate and adapt to the congested network or slow re- 
ceiver. 

Performance issues with aggregation: TCP was 

designed based on the fact that the out-of-order se- 
quence is generally a good indicator of lost packets or 
congested network. However, such assumption no longer 
holds in BaPu. 

• Out-or-order packets: In BaPu, the packets be- 
longing to the same TCP session are intention- 
ally routed through multiple BaPu-^4Ps via di- 
verse backhaul connections in terms of capacity, 
latency, traffic load, etc. This results in serious 
out-of-order sequence at BaPu -Gateway , which 
eventually injects the out-of-order packets to the 
Receiver. 

• Double RTT: Also, due to the aggregation proto- 
col, data packets in BaPu are delivered to the Re- 
ceiver with a double round-trip-time (RTT) com- 
pared to a regular link. This causes the Sender's 
CWND to grow more slowly and peak at lower 
values. 

Consequently, with an unplanned aggregation method, 
the TCP congestion control mechanism is falsely trig- 
gered, resulting in considerably low throughput. As we 
show later in Section 4, a simplified prototype of our 
system, which share similarities with the system in [15], 
gives poor TCP throughput. 

Solution: To address the performance issue, we first 
investigated a simple approach: data packets forwarded 
by BaPu-t4Ps are buffered at BaPu- Gateway until a 
continuous sequence is received before injecting to the 
Receiver. This solution, however, encounters the fol- 
lowing issues: 1) Efficiency: Introducing a buffer for 
each session at BaPu -Gateway is wasteful of memory, 
since the Receiver already maintains a TCP buffer for 
its own session. Furthermore, this does not scale well 
when more simultaneous sessions are established. 2) 
Optimality: Due to the difference in capacity, latency, 
loss rate among backhaul uplinks, it is not clear how to 
determine the optimal buffer size. 3) Performance: In 
fact, we implemented a buffering mechanism at BaPu- 
Gateway, and the results (Section 4.2) showed that us- 
ing buffering mechanism does not help improving the 
TCP throughput. 

Now we introduce a novel mechanism called Proactive- 
ACK^ which is used in step 7 of BaPu protocol. The 
principle of Proactive-ACK mechanism is to actively 
control the exchange of acknowledgements instead of re- 
lying on the default behaviour of the end-to-end session. 
With Proactive-ACK, we solve both out-of-order packet 
and double RTT issues. In the following paragraphs, we 
call acknowledgements actively sent by BaPu -Gateway 
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spoofed acknowledgements, while the ones sent by the 
Receiver are real acknowledgements. 

Managing DUPACK: In BaPu, most of out-of-order 
packets are caused by the aggregation mechanism via 
multiple BaPu-t4Ps. To avoid the cutting off of the 
CWND at the Sender, we intentionally discard all DU- 
PACKs received from the Receiver, as we observed that 
most of DUPACKs generated by the Receiver in BaPu 
are due to the multiple-path aggregation. 

However, by dropping DUPACKs from the Receiver, 
we need to handle the case of actual lost packets in the 
air between the Sender and BaPu-^4Ps. Concretely, if 
the report for expected TCP sequence number is not 
received within certain time window, it is implied that 
this sequence is lost on all participating BaPu-^Ps. 
Now that BaPu -Gateway sends a spoofed DUPACK 
back to the Sender in order to mimic the TCP fast 
retransmission mechanism for fast recovery from packet 
loss. 

Managing ACK: Besides the effect of DUPACKs, the 
CWND size of the Sender is also highly affected by the 
double RTT introduced by the protocol. Not only the 
CWND grows slowly, but the chance of CWND being 
cut off is also higher. With Proactive- ACK mechanism, 
in step 7, BaPu -Gateway sends back to the Sender the 
spoofed ACK after receiving the report from BaPu- 
APs. The intuition is that all the packets that are 
reported by some BaPu-^4Ps are currently stored in 
those BaPu-t4Ps' buffer. Due to the reliability of the 
TCP tunnel between BaPu-^Ps and BaPu- Gateway, 
the reported packets will be eventually forwarded to 
BaPu -Gateway in reliable manner. Therefore, as long 
as BaPu -Gateway identifies a continuous range of re- 
ported TCP sequence, immediately sending a spoofed 
ACK back to the Sender helps maintaining a high and 
constant throughput, as the RTT with respect to the 
Sender is reduced to a value around the real RTT. This 
approach prevents the cutting off of CWND at the Sen- 
der. 

Since BaPu- Gateway manually sends spoofed ACKs 
to the Sender, on reception of real ACKs the Receiver, 
BaPu -Gateway simply discards the real ACKs. 

TCP semantics: We have two important remarks on 
the TCP semantics: 

• Immediately sending the spoofed ACKs after re- 
ceiving the reports may result in spoofed ACKS 
being received at the Sender before data packets 
being forwarded to the Receiver. This increases 
the CWND in a more aggresive manner than the 
standard mechanism. 

• Dropping real DUPACKs and sending spoofed DU- 
PACKS can increase the time for recovery of an 
actual loss of packet, because the loss reflected by 
the Receiver is not immediately signaled to the 
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Figure 6: BaPu Experiment Setup. 7 BaPu-APs and 
1 BAPu-Gateway are inter-connected. A traffic shaping 
box is set up in between to emulate a typical residential 
network setting. 

Sender. For example, if an AP who has been 
scheduled to forward the selected packet is sud- 
denly offline, it takes a longer time for the packet 
to be scheduled again after a timeout and later 
forwarded to the Receiver. 

Despite the slightly difference in TCP semantics, the 
Proactive- ACK mechanism has been proved to give a 
significant improvement to the TCP throughput. We 
present these results in Section 4. 

3.4 Scheduling 

The bandwidth aggregation performance depends on 
the efficiency of multiplexing data among BaPu-^4Ps 
to best utilize the idle uplink bandwidth. 

In BaPu, we adopt a centralized scheduler at BaPu- 
Gateway. There are two main factors to select this 
design. First, with the centralized design, it does not 
only simplify the implementation, but also allow easy 
extension of the design with extra logic to further opti- 
mize the scheduling strategy. Second, a scheduler usu- 
ally requires complex processing and memory capabil- 
ity, which might overload the BaPu-^4Ps with much 
lower capability if scheduling decisions are migrated to 
the APs. 

The scheduling strategy is based on the received re- 
ports in step 6 and 7 of the protocol. Each report from a 
BaPu-^4P contains a sending buffer size obtained from 
the Linux kernel via ioctlO function call. This value 
specifies how much a BaPu-^4P can contribute to the 
aggregation. Based on reports, BaPu- Gateway applies 
First-Come-First-Served strategy to select a forwarder 
among BaPu-^4Ps who have captured the same packet. 
This approach is similar to those applied in [15, 16]. The 
rationale for choosing this approach are 

• Fairness: Sharing bandwidth for aggregation takes 
into account the available bandwidth of participat- 
ing BaPu-^4Ps, as because AP owners have differ- 
ent subscription plans. 

• Protocol independence: Scheduling decision is made 
based on the BaPu-^Ps' sharing capability, not 
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9(b) UDP and TCP aggregation efficiency 
Figure 7: BaPu aggregation for UDP and TCP with 2Mbps 32ms RTT uplinks. 



Distance 


RTT 


Regional: 500 - 1,000 mi 


32ms [1] 


Cross-continent: ~ 3,000 mi 


96ms [1] 


Multi-continent: ~ 6,000 mi 


192ms [1] 


inter- AP in Boston 


20ms ~ 80ms 



max UDP 


1.94 Mbps 


max BaPu UDP 


1.82 Mbps 


max TCP 


1.9 Mbps 


max BaPu TCP 


1.8 Mbps 



Table 3: Network RTT Latency. Inter-AP RTT 
measured by our Open Infrastructure WiFi testbed in 
greater Boston, representing typical RTT between home 
APs, covering Comcast, RCN, and Verizon [24]. 



on the particular transport protocol. 

4. EVALUATION 

In this section, we evaluate the performance of BaPu 
for UDP and TCP in various system settings. 

Experiment Setup: Our experiment setup is shown 
in Figure 6. Our testbed consists of a Sender, 7 BaPu- 
t4Ps, a BaPu -Gateway , a Receiver node, and a traffic 
shaping box. All APs are Buffalo WZR-HP-G300NH 
802.1 In wireless routers. This router model has a 400MHz 
CPU with 32MB RAM. We rehashed the APs with 
OpenWRT firmware, running Linux kernel 2.6.32 and 
ath9k WiFi driver. In our experiments, we select one 
BaPu-^4P to act as a Home-AP which the Sender is 
always associated to. The other 6 BaPu-^4Ps act as 
Monitor-APs to capture the traffic in monitor mode. 
The BaPu -Gateway runs on a Linux PC, and the Re- 
ceiver runs behind the BaPu- Gateway. The Sender 
and the Receiver are both laptops with 802.1 In WiFi 
card, running the standard Linux TCP/IP stack. 

To emulate traffic shaping as with residential broad- 
band, we use the traffic shaping box between the BaPu- 
^4Ps and BaPu- Gateway. We use Linux' iptables and 
tc with the htb module to shape the downlink band- 
width to 20Mbps and the uplink to 2Mbps. Also, to em- 
ulate network latency between BaPu-^4Ps and BaPu- 
Gateway, we use net em to shape the RTT with different 
values. The bandwidth and latency parameter are se- 



Table 4: Maximum theoretical goodput for UDP and 
TCP with and without BaPu overhead. Data pay load 
size is 1350Bytes. Uplink capacity is 2Mbps. 



lected to represent the typical bandwidth capacity and 
regional latency in residential cable broadband that we 
have measured in Boston's urban area (Table 3). 

In our experiments, we issue long-lived 30 minutes 
iperf flows (both TCP and UDP) from Sender to Re- 
ceiver. We choose 1350Byte as TCP/UDP payload size 
in our iperf test to make sure that the whole client 
IP packet can be encapsulated in one IP packet while 
an BaPu-^P sends it through its TCP tunnel. All 
throughput values reported in our experiment are the 
iperf throughput, which is the goodput. 

In the evaluation, we compare throughput of UDP 
and TCP in different system scenarios. More precisely, 
we evaluate the following scenarios: 

• BaPu: BaPu system without any buffering or 
Proactive- ACK mechanism. 

• SimpleBuffer: BaPu system without Proactive- 
ACK mechanism, but enhanced by buffering at 
BaPu -Gateway. 

• BaPu-PrO: this is the full BaPu system. 

4.1 B APU: Efficient UDP, Poor TCP 

4.1.1 System efficiency with UDP throughput 

The practicality of BaPu lies in its efficiency. In con- 
trast to related work, BaPu's transparency goal, not 
requiring any modifications at the client side, has moti- 
vated the design of BaPu's underlying technical details. 
We now first measure BaPu's efficiency by the through- 
put with UDP, as it provides a light-weight end-to-end 
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Figure 8: Sender's TCP CWND growth compared be- 
tween BaPu and regular single AP with 32ms RTT and 
total 14Mbps uplink. 

transmission between Sender and Receiver. Figure 7a 
shows the achieved aggregated UDP throughput with 
numbers of participating BaPu-^LPs increasing from 1 
to 7. We observe that the aggregated UDP through- 
put increases proportionally with the number of BaPu- 
APs, and achieves 12.4Mbps with 7 BaPu-^Ps. To 
put this figure into perspective, note that related work 
by Jakubczak et al. [15] achieves similar UDP through- 
put but without support for TCP or client transparency. 

4.1.2 Low TCP throughput 

We conduct the same experiments also for TCP trans- 
mission. Figure 7a shows that the aggregated TCP 
throughput does not benefit much when the number of 
BaPu-t4Ps increases. The TCP aggregated throughput 
is always lower than the UDP's in the same setup, and 
the gap between UDP and TCP performance increases 
along with the number of BaPu-^Ps. For example, we 
achieve only 6.83Mbps with 7 BaPu-^Ps. 

4.1.3 Aggregation efficiency 

In addition to measuring aggregated throughput, we 
evaluate our system based on another metric, aggre- 
gation efficiency. We define aggregation efficiency as 
the ratio between practical throughput over the maxi- 
mum theoretical goodput. Due to the TCP/IP header 
and BaPu protocol overhead, the actual goodput is 
less than the uplink capacity. With all protocol header 
overhead accounted, we derive the maximum theoreti- 
cal goodput as the given backhaul capacity of 2Mbps. 
Table 4 lists the maximum throughput when data is 
transmitted via standard UDP/TCP and via BaPu. 

As shown in Figure 7b, BaPu UDP can harness close 
to 100% idle bandwidth. Even if we consider the extra 
overhead incurred by BaPu protocol messages, UDP 
aggregation efficiency is still over 90% in all cases. In 
contrast, the aggregation efficiency for TCP degrades 
quickly as more BaPu-^4Ps join the cooperation. With 
7 BaPu-^Ps, BaPu transforms only 50% of idle band- 
width to effective throughput. 

4.1.4 Discussion: BaPu's poor TCP performance 
We can observe several factors in Section 3.3 that de- 




Number of APs 

Figure 9: BaPu vs. SimpleBuffer comparison in 
TCP throughput with 2Mbps 32ms RTT uplinks. 

crease the aggregated TCP throughput. In this section, 
we carry out an analysis on the Sender's CWND size in 
BaPu. To justify our analysis, we inspect the TCP be- 
havior by examining the Linux kernel TCP stack vari- 
ables. We call getsockoptO to query the TCP_INF0 
Linux kernel data structure. TCP_INF0 includes the 
system time stamp, the Sender's CWND, number of 
retransmissions, etc. We have modified the iperf code 
to log TCP_INF0 each time iperf calls send() to write 
the application data to the TCP socket buffer. 

Figure 8 shows the CWND growth in a 120 second 
iperf test with 7 BaPu-^Ps. The theoretical through- 
put here 2Mbps x 7 = 14Mbps. In comparison, we carry 
out another iperf test with standard TCP through a 
single, regular AP with 14Mbps uplink capacity. The 
CWND growth in a normal TCP connection is also 
shown in Figure 8. As shown, the Sender's CWND 
remains at a very low level. Our captured packet trace 
at the Sender shows that lots of DUPACK packets and 
RTO incur a lot of retransmissions, which results in very 
low TCP throughput. 

4.2 Does SimpleBuffer help? 

As discussed in Section 3.3, a simple buffering mech- 
anism does not solve the TCP performance issue due to 
difference in BaPu-^4P uplink characteristics (latency, 
packet loss). In this section, we show experimentally 
that a buffering mechanism cannot help in improving 
the TCP throughput. The experiment is performed for 
equal uplink capacity and latency, i.e., we eliminate ex- 
ternal factors such as asymmetric links among BaPu- 
APs. 

Figure 9 depicts the throughput comparison between 
BaPu and SimpleBuffer. Surprisingly, the through- 
put is worse with SimpleBuffer. We have also ad- 
justed the buffer size at BaPu -Gateway , but the through- 
put still remains as low as shown in Figure 9. We have 
investigated Sender's CWND, and we have seen that it 
peaks at low values, similarly to the behavior in BaPu. 
The packet trace also shows a lot of retransmissions. 

4.3 BAPu-PRO Performance 
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Figure 10: BaPu-Pro vs. BaPu: comparison with 2Mbps 32ms RTT uplinks. 
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Figure 11: TCP sender CWND growth comparison: BaPu-Pro vs. BaPu vs. normal TCP. 



Now, we conduct a comprehensive set of experiments 
to evaluate the performance of BaPu- Pro. First, we 
validate our Proactive- ACK mechanism by comparing 
BaPu-Pro against BaPu. Second, we measure the 
performance of BaPu-Pro under a variety of network 
settings (network latency, wireless link quality, etc.). 
Finally, we demonstrate that BaPu-Pro is feasible for 
both, streaming and large file transfer applications. 

4.3.1 TCP Throughput - BaPu-Pro vs. BaPu 

We carry out the same iperf test as described in 
Section 4.1 with BaPu-Pro. As shown in Figure 10a, 
the aggregated TCP throughput of BaPu-Pro signif- 
icantly outperforms the one of BaPu. With 7 BaPu- 
APs, BaPu-Pro achieves 11.04Mbps, i.e., 62% im- 
provement over BaPu. Furthermore, Figure 10b shows 
that BaPu-Pro achieves at least 88% aggregation ef- 
ficiency in our setup, and it achieves at least 83% of 
the upper limit of standard TCP throughput. These 
results demonstrate that BaPu-Pro can achieve high 
aggregated throughput with high aggregation efficiency 
for TCP in practical settings. 

4.3.2 Proactive- ACK benefit 

To justify our Proactive- ACK mechanism, we adopt 
the same method as in Section 4.1.4 to examine the 
TCP CWND growth. Figure 11 shows that BaPu- 
Pro allows the CWND to grow to very high values, 
contributing to the high throughput. For convenience, 
we also run a regular TCP session with a throttled band- 
width 11Mbps (similar to the BaPu-Pro's resulted through- 
put). The CWND growth for BaPu-Pro and regular 



TCP shares a similar pattern, which implies that our 
design and implementation can efficiently and transpar- 
ently aggregate multiple slow uplinks. 

4.3.3 Impact of network latency 

For TCP transmissions, RTT is an important factor 
that has impact on the throughput. In another experi- 
ment, we measure the performance of BaPu with 4 dif- 
ferent network latency settings listed in Table 3. Each 
latency represents certain application scenarios. For ex- 
ample, when users upload HD video with BaPu to CDN 
edge servers, the latency to CDN servers is generally 
the regional latency (32ms). When users upload data 
to their friends in another continent, the RTT is on av- 
erage 192ms. Besides, according to our measurements 
in a residential WiFi testbed [24], we observe that the 
latency between broadband subscribers may vary con- 
siderably, ranging between 20ms and 80ms. This de- 
pends on whether users are with the same or a different 
ISP. Consider the case that a user uploads data with 
BaPu through neighboring APs to another user in the 
same city, the latency between the BaPu-^4Ps and the 
end user can be quite different. 

Given a certain number of APs, we assign to each 
BaPu -AP a random RTT value between 20ms and 80ms. 
We carry out this test for 10 runs and report the av- 
erage throughput. As shown in Figure 12a, BaPu- 
Pro throughput slightly declines as network latency in- 
creases. In random latency setting, the resulted through- 
put shows no significant difference. 

4.3.4 Impact of lossy wireless links 
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Figure 12: BaPu-Pro TCP throughput. 
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Figure 13: Instantaneously received throughput com- 
parison: 11Mbps Streaming vs. Unlimited rate. 



The wireless links in a real neighbourhood can be 
very lossy for a variety of reasons, such as cross chan- 
nel interference and distant neighboring APs. Besides, 
since Monitor-APs switch between transmit and receive 
mode, they cannot overhear all transmitted packets. To 
estimate the potential of BaPu highly lossy wireless 
environments, we emulate packet loss at Monitor-APs 
by dropping received packets with a probability P. No 
losses were inflicted on Home-AP, because Sender car- 
ries out unicast to Home-AP, and 802.11 MAC already 
handles packet loss and retransmissions automatically. 
We conduct the experiment with 3 values of P: 20%, 
40%, and 60%. 

As indicated in Figure 12b, the throughput reduction 
on lossy wireless links is very limited in all cases. The 
good performance can be explained by the link diversity 
combined with the centralized scheduling mechanisms. 
The probability of some packet not overheard by at least 
one Monitor-AP is negligible small, especially in case 
of high number of participating APs. This also explains 
why 7 BaPu-^4Ps achieve higher throughput with P = 
60% than with P = 20%. 

4.3.5 Streaming vs. large file transfer 

One important goal in BaPu's design is to support 
instant sharing of high-bitrate HD videos directly be- 
tween users using streaming. The motivation behind 
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Figure 14: Regular user download behaviour with and 
without background upload traffic. No impact is appar- 
ent in the presence of competing background traffic. 

is that today the major online streaming services (e.g., 
Netflix) run on TCP based streaming technologies, such 
as HTTP based Adaptive Bitrate Streaming. Real time 
streaming generally requires stable instantaneous through- 
put. In this experiment, we study the potential of BaPu 
as a solution to high-bitrate real-time streaming. 

To emulate the streaming traffic, we use nuttcp to 
issue a TCP flow with a fixed 11Mbps sending rate. 
Figure 13 shows Receiver's instantaneous throughput 
in a 100 second session. Receiver achieves a reasonably 
stable throughput in the whole session. It indicates that 
BaPu can sustain high-bitrate streaming through ag- 
gregated uplinks. In comparison, the iperf flow with 
unlimited sending rate shows much higher fluctuation. 

5. IMPACT OF UPLINK SHARING 

BaPu is essentially a crowd-sourcing approach that 
shares users' idle bandwidth to help others. The goal 
is to harness as much idle bandwidth as possible with 
minimal impact on home users' network usage. We first 
show that with standard Linux traffic shaping tools, 
bandwidth sharing has minimal impact on regular traf- 
fic. Next, we study how much bandwidth can be har- 
nessed with a testbed in residential broadband. 

5.1 Prioritizing Home User's Traffic 
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Figure 15: Regular uplink observation with and without 
background traffic. The low-priority background traffic 
backs off as soon as regular traffic starts. 
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Figure 16: Fraction of time spent by each AP with cer- 
tain background upload throughput over the AP's total 
online time. 



Techniques and tools for traffic shaping, such as Linux's 
tc are widely available. While a full-fledged system may 
use complicated traffic prioritization, it is sufficient for 
BaPu to simply classify traffic in two classes: regular 
user traffic with the highest priority, and background 
sharing traffic with a minimum bandwidth guarantee, 
but allowed to grow if more bandwidth is available. To 
implement this, we use Hierarchical Token Bucket and 
Stochastic Fair Queuing modules within tc to fairly dis- 
tribute traffic belonging to the same class. 

We set up the traffic shaping on one OpenWRT home 
router, and validate the correctness of the traffic shap- 
ing with two tests. In the first test, we first generate reg- 
ular download traffic for 10 minutes to obtain a baseline 
of AP's download throughput. After the baseline mea- 
surement, we start the background upload for 20 min- 
utes, emulating the uplink sharing. As the upload goes 
on, we relaunch the regular download traffic. As shown 
in Figure 14, the user's regular download throughput 
is not affected, because the TCP ACKs related to the 
user download have been prioritized. Also, TCP ACKs 
consume limited uplink bandwidth, and thus has negli- 
gible impact to the background upload. In the second 
test, we examine the analogous case for regular upload 
traffic. We first start emulated background upload with 
a minimum bandwidth guarantee of 500Kbps. During 
the background upload, we start the regular user up- 
load. As shown in Figure 15, the regular upload traffic 
takes over more bandwidth, while the background up- 
load backs off properly (but not lower than 500Kbps). 
We conclude that with proper traffic shaping, BaPu 
can provide uplink sharing with a minimal impact on 
users' regular traffic. 

5.2 Push Uplink Sharing to the Limit 

To find out how much idle uplink bandwidth can be 
harnessed, we instrument an uplink throughput exper- 
iment to 17 residential APs, covering 2 major ISPs in 
Boston (Table 5). Each AP is configured with proper 
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Figure 17: Duration of background traffic flow of certain 
throughput classes. 

traffic shaping. With constant background upload, each 
AP reports background upload throughput every 10 sec- 
onds, lasting for 16 days. 

Table 5: Uplink experiment data summary 



Home APs 
Data collection time 
Mean AP online time 
Throughput samples 



Comcast (14), RCN (3) 
May 22 - Jun 13, 2012 
381 hours (~ 16 days) 
2.3 millions 



As shown in Figure 16, all APs can share l-3Mbps 
uplink throughput during most of the experiment time. 
Each AP's mean shared throughput is very close to its 
uplink limit set by ISP. Also, we investigate how long 
a certain upload throughput can last. Figure 17 shows 
the the CDF of duration for which the background up- 
load can sustain certain throughput during peak hours 
(18pm~23pm). We see that, there is over 80% chance 
that the background upload throughput can stay above 
2Mbps for over 30 minutes. To conclude, there is abun- 
dant idle uplink bandwidth in residential broadband 
that can be exploited for upload sharing. 

6. RELATED WORK 

BaPu system is inspired by design principles of sev- 
eral earlier protocols, it however addresses unique con- 
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straints and goals and results in a novel combination of 
techniques that achieves high efficiency. Several earlier 
research works proposed to improve the performance 
of TCP over wireless links by using intermediate nodes 
to assist in the recovery of lost packets include Snoop 
TCP [5], and Split TCP for ad hoc networks [17]. Mul- 
tiple radio links to improve devices throughput have 
also been explored from several perspectives including 
traffic aggregation [16], mutipath forwarding [15], mit- 
igation of wireless losses [21, 22]. In addition to sys- 
tems that rely on multiple radio interfaces [4], many 
solutions and algorithms were proposed for a single ra- 
dio interface that carefully switches across multiple ac- 
cess points while providing the upper layers of the net- 
work stack a transparent access through a virtual in- 
terface [8, 16, 20, 31]. Solutions to overcome the lim- 
ited APs backhaul through aggregation using a virtu- 
alized radio interface include the initial Virtual- WiFi 
system where two TCP connection might be services by 
through two different APs [20], FatVAP that achieves 
fast switching a smart AP selection [16], ARBOR that 
add security support [31], Fair WLAN that provides 
fairness [12]. Many of these systems require techniques 
for fast switching across access points to reduce the im- 
pact on TCP performance in terms of delay and packet 
loss as proposed in Juggler [23], and WiSwitcher [11]. 
In [28], an analytical model is proposed to optimze con- 
current AP connections for highly mobile clients. They 
also implement Spider, a multi-AP driver using optimal 
AP and channel scheduling to improve the aggregated 
throughput. Unlike BaPu, these works do not aggregate 
the throughput for single transport layer connection, 
which is critical for client transparency. Divert [22] pro- 
pose a central controller to select the optimal AP across 
multiple BSes in WLANs in order to reduce the path- 
dependent downlink loss from AP to client. Rather than 
improving the wireless link quality, BaPu is aimed to ag- 
gregate the wired capacity behind APs. In BaPu, the 
sender communicates with its home AP as usual. How- 
ever, BaPu does benefit from the link diversity across 
APs while aggregating the backhaul capacity. ViFi [6] 
propose a probabilistic algorithm for coordination be- 
tween basestations to improve the opportunistic con- 
nectivity of the client-BS communication. Similar to 
Divert, ViFi is not for aggregating throughput. Also, 
in section 3.4, we show that such probabilistic solution 
sets limitations on throughput aggregation. The closest 
approach to our work is the Link-alike system where 
access points coordinate to opportunistically schedule 
the traffic over their backhaul links [15]. Our approach 
differs from previous work in requiring that the client 
devices remain unchanged (unicast connection to the 
AP) and transparently supports protocols like TCP. 

Being completely transparent to the clients and con- 
straining each link AP-Destination flow to be TCP- 



friendly makes efficient multipath transport, a key com- 
ponent of our system. There has been a significant 
amount of work in this area for quite some time from 
various perspectives that are fairly different from our 
setup. Previous work identified the issues with differ- 
ential delay, and rates and mostly focussed on provid- 
ing reliability, flows balancing, and maintaining fair- 
ness. Proposed solutions, require the modification of 
the client devices network stacks, and usually do not 
aim at increasing capacity through simultaneous use of 
all paths. For example, the IETF has two standards 
on transport protocols using multipath. The Stream 
Control Transmission Protocol (SCTP) was primarily 
designed for multi-homed devices that require fail-over 
support [29], the more recent Multi-Path TCP (MPTCP) 
is a TCP extension that aims at enabling nodes to effi- 
ciently communicate utilizing multiple parallel paths [10]. 
In recent work, [30] proposed a congestion control mech- 
anism for MPTCP with the objective of reliability and 
fairness. Other transport protocols that require the 
modification of the client devices, include pTCP [13] an 
end to end transport protocol that achieves bandwidth 
aggregation on multi-homed mobile hosts, RCP [14] a 
Receiver Centric Protocol that aggregates heterogeneous 
interfaces traffic and carries link specific congestion con- 
trol, R-MTP [18] balances and coordinates the traffic 
over wireless links with varying characteristics, Hori- 
zon [26] uses back-pressure technique to balance the 
traffic across forwarding paths. Beyond mobile com- 
munication environments, multipath TCP features have 
also been finding applications in various networking en- 
vironments such as data [7, 27]. The distinguishing 
element of BaPu is that it aims at transparently sup- 
porting unmodified client devices and TCP/IP stacks 
while efficiently aggregating the APs backhauls. 

7. CONCLUSION 

In this work, we present the design and implemen- 
tation of BaPu, a complete software based solution on 
WiFi APs for aggregating multiple broadband uplinks. 
First, with a large scale wardriving data and a long term 
measurement in Boston residential broadband, we show 
that the high AP density and under utilized broadband 
uplinks calls for a solution to harness the idle bandwidth 
for a boosted uplink throughput. 

With our client transparent design, BaPu offers generic 
support for legacy client and a large variety of network 
applications. However, the client transparency design 
raises many new technical challenges. In particular, we 
propose a novel mechanism, called Proactive- ACK, to 
address the challenge of multiplexing single TCP ses- 
sion through multiple paths without degrading perfor- 
mance. The benefit of such mechanism is analysed with 
experimental data. We carry out an extensive set of 
experiments to evaluate the BaPu throughput perfor- 
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mance for both UDP and TCP, in a variety of realistic 
network settings. BaPu achieves over 95% aggregation 
efficiency for UDP and over 88% for TCP, even in lossy 
wireless environment. 

Besides, to further justify the feasibility of BaPu as 
a crowd-sourcing mechanism, we empirically show the 
potential idle uplink bandwidth that can be harnessed 
from residential broadband networks. We also provide 
a design guideline for such bandwidth sharing system to 
eliminate the negative impact to home users. Also, the 
software based solution makes BaPu an easy incremen- 
tal deployment, especially as APs are becoming social 
and cloud-managed. 
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